{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### 作业三：\n",
    "写一个可以将MNIST图片向任意方向（上，下，左，右）移动一个像素功能。然后对训练集中的每张图片，创建四个位移后的副本，每个方向一个，添加到训练集。最后，在这个扩展过的训练集上训练模型，衡量其在测试集上的精度，来优化精度，这种人工扩展训练集的技术成为数据增广或训练集扩展 15分"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### 思路\n",
    "- 获取数据:done\n",
    "- 获取样本X,标签y；将X，y的顺序随机打乱：done\n",
    "- 获取训练集60000；测试集10000：done\n",
    "- 获取一个样本数据：39000：？，之后测试用：done\n",
    "- 样本数据shape(28,28),(上下左右)移动一个像素\n",
    "- 训练集都移动一个像素\n",
    "- 位移副本添加到训练集：np.c_\n",
    "- 使用模型：\n",
    "    - SGD梯度下降 分类器\n",
    "    - 支持向量机svm\n",
    "    - 随机森林和朴素贝叶斯处理多类别分类器（这个例子不使用）\n",
    "- 交叉验证取得precision/recall\n",
    "- "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [],
   "source": [
    "from sklearn.datasets import fetch_mldata\n",
    "import numpy as np\n",
    "import pandas as pd \n",
    "import matplotlib.pyplot as plt"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [],
   "source": [
    "# 数据集读取\n",
    "mnist = fetch_mldata('mnist-original', data_home='./')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [],
   "source": [
    "# 数据\n",
    "X=mnist['data']\n",
    "y=mnist['target']"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [],
   "source": [
    "# 洗牌 \n",
    "shuffle_index=np.random.permutation(70000)\n",
    "# 数据\n",
    "X=X[shuffle_index]\n",
    "# 标签\n",
    "y=y[shuffle_index]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "(70000, 784) (70000,)\n"
     ]
    }
   ],
   "source": [
    "print(X.shape,y.shape)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [],
   "source": [
    "# 训练集，测试集拆分\n",
    "X_train,y_train,X_test,y_test=X[:60000,:],y[:60000],X[60000:,:],y[60000:]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# data_new=X_train.reshape((len(X_train),int(np.sqrt(X_train.shape[1])),-1))\n",
    "# data_new.shape"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [],
   "source": [
    "import matplotlib"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAPsAAAD4CAYAAAAq5pAIAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+j8jraAAANcElEQVR4nO3db6hc9Z3H8c9nsy2KKRjNTQypeLvFwMqqaR3CgmvoUrZGwT8FGxowyYKYPlBIoQ/U7oMq+CAu29Z9sBTSTWiydi0XUtEHEiuhKM2D4ihZEzeoWbmbpInJjUFjH0g1/e6De7LcxDtnxjln5hzzfb/gMjPnO2d+X4Z8cmbmN2d+jggBuPj9RdMNABgPwg4kQdiBJAg7kARhB5L4y3EOtnjx4picnBznkEAq09PTOnXqlOerVQq77TWS/lXSAkn/HhFbyu4/OTmpbrdbZUgAJTqdTs/a0C/jbS+Q9G+SbpN0naR1tq8b9vEAjFaV9+yrJB2KiHci4k+SfiXprnraAlC3KmFfLunInNtHi23nsb3Jdtd2d2ZmpsJwAKqoEvb5PgT41HdvI2JrRHQiojMxMVFhOABVVAn7UUlXz7n9ZUnHqrUDYFSqhP0VSdfa/ortL0r6rqTn6mkLQN2GnnqLiE9sPyjpBc1OvW2PiDdq6wxArSrNs0fE85Ker6kXACPE12WBJAg7kARhB5Ig7EAShB1IgrADSRB2IAnCDiRB2IEkCDuQBGEHkiDsQBKEHUhirD8l/Xl26NChnrU77rijdN8333yz7nbOU7Y455IlS0r3feKJJ0rr99xzT2l94cKFpXW0B0d2IAnCDiRB2IEkCDuQBGEHkiDsQBKEHUiCefYBlc2ll83BS9KCBQvqbuc8Z8+e7Vk7ffp06b73339/af36668vrd90002ldbQHR3YgCcIOJEHYgSQIO5AEYQeSIOxAEoQdSIJ59gGVnZM+6nn0Jq1du7a0vmvXrtL6ypUr62wHFVQKu+1pSR9KOivpk4jo1NEUgPrVcWT/+4g4VcPjABgh3rMDSVQNe0j6je1XbW+a7w62N9nu2u7OzMxUHA7AsKqG/eaI+Lqk2yQ9YHv1hXeIiK0R0YmIzsTERMXhAAyrUtgj4lhxeVLSM5JW1dEUgPoNHXbbl9n+0rnrkr4l6UBdjQGoV5VP45dKesb2ucf5z4jYXUtXLbRz586etc2bN5fue+bMmbrbGZvDhw+X1qempkrrzLO3x9Bhj4h3JN1YYy8ARoipNyAJwg4kQdiBJAg7kARhB5Jw2XK/det0OtHtdsc23ri88MILpfUTJ06MdPy9e/f2rG3fvn2kY/fz8ccfNzp+Np1OR91u1/PVOLIDSRB2IAnCDiRB2IEkCDuQBGEHkiDsQBL8lHQNbr311kbH37BhQ8/aW2+9VbrvSy+9VHc7aCmO7EAShB1IgrADSRB2IAnCDiRB2IEkCDuQBPPsF7nip757upiXm8b5OLIDSRB2IAnCDiRB2IEkCDuQBGEHkiDsQBKEHUiib9htb7d90vaBOduusP2i7beLy0WjbRNAVYMc2X8hac0F2x6WtCcirpW0p7gNoMX6hj0iXpZ0+oLNd0naUVzfIenumvsCULNh37MvjYjjklRcLul1R9ubbHdtd2dmZoYcDkBVI/+ALiK2RkQnIjoTExOjHg5AD8OG/YTtZZJUXJ6sryUAozBs2J+TtLG4vlHSs/W0A2BU+p7PbvtpSd+QtNj2UUk/krRF0pTt+yQdlvSdUTaJcu+++27P2vvvvz/GTtBmfcMeEet6lL5Zcy8ARohv0AFJEHYgCcIOJEHYgSQIO5AEPyV9Edi9e3fP2v79+8fYCdqMIzuQBGEHkiDsQBKEHUiCsANJEHYgCcIOJME8+0Vg7969PWtnz54dYydoM47sQBKEHUiCsANJEHYgCcIOJEHYgSQIO5AE8+yfA1u2bCmtb9u2rWdtwYIFdbdznoceemikj9+Usp/nlqSPPvqo0uMvXry4Z23hwoWVHrsXjuxAEoQdSIKwA0kQdiAJwg4kQdiBJAg7kATz7C0wPT1dWp+amhpPI0NYsWJFY2M/+eSTpfUPPvigZy0iSvfduXNnaf3IkSOl9X7KvhuxYcOGSo/dS98ju+3ttk/aPjBn26O2/2B7X/F3+0i6A1CbQV7G/0LSmnm2/zQiVhZ/z9fbFoC69Q17RLws6fQYegEwQlU+oHvQ9uvFy/xFve5ke5Ptru3uzMxMheEAVDFs2H8m6auSVko6LunHve4YEVsjohMRnYmJiSGHA1DVUGGPiBMRcTYi/izp55JW1dsWgLoNFXbby+bc/LakA73uC6Ad3G++0fbTkr4habGkE5J+VNxeKSkkTUv6XkQc7zdYp9OJbrdbqeHPo37z6HfeeWdp/eDBg6X1st+GH/X57FdeeWVp/dJLL+1Zs11p7GPHjpXWy56Xfr+n3+Tz1u9c+jKdTkfdbnfeJ7bvl2oiYt08m3t/IwBAK/F1WSAJwg4kQdiBJAg7kARhB5LgFNcx2LFjR2m939Ram7333nul9SanBdvs7rvvHvuYHNmBJAg7kARhB5Ig7EAShB1IgrADSRB2IAnm2Wvw1FNPldYfe+yxkY5fdppyv1M5R62tvfU7tbtqb1dddVVp/d577630+MPgyA4kQdiBJAg7kARhB5Ig7EAShB1IgrADSTDPPqDdu3f3rG3evLl031Gft93mc8bb2lu/efTly5eX1teuXVta77fs8o033lhaHwWO7EAShB1IgrADSRB2IAnCDiRB2IEkCDuQBPPsAypbRvfMmTNj7ATnTE5OltbXr18/9GNv3LixtH7NNdcM/dhN6Xtkt3217d/aPmj7Ddubi+1X2H7R9tvF5aLRtwtgWIO8jP9E0g8i4q8l/a2kB2xfJ+lhSXsi4lpJe4rbAFqqb9gj4nhEvFZc/1DSQUnLJd0l6dy6RjskjX89GwAD+0wf0NmelPQ1Sb+XtDQijkuz/yFIWtJjn022u7a7MzMz1boFMLSBw257oaRdkr4fEQN/IhURWyOiExGdiYmJYXoEUIOBwm77C5oN+i8j4tfF5hO2lxX1ZZJOjqZFAHXoO/Vm25K2SToYET+ZU3pO0kZJW4rLZ0fSYUuU/TTw6tWrS/ednp4urR8+fHiYlgZyww03lNYvv/zykY0tlf9k8yOPPFK674oVK0rrl1xySWl96dKlpfVsBplnv1nSekn7be8rtv1QsyGfsn2fpMOSvjOaFgHUoW/YI+J3ktyj/M162wEwKnxdFkiCsANJEHYgCcIOJEHYgSQ4xXVAa9asGaom9V+y+fHHHx+qp3NuueWWnrWpqanSfZmLzoMjO5AEYQeSIOxAEoQdSIKwA0kQdiAJwg4k4bLzjevW6XSi2+2ObTwgm06no263O+9ZqhzZgSQIO5AEYQeSIOxAEoQdSIKwA0kQdiAJwg4kQdiBJAg7kARhB5Ig7EAShB1IgrADSRB2IIm+Ybd9te3f2j5o+w3bm4vtj9r+g+19xd/to28XwLAGWSTiE0k/iIjXbH9J0qu2XyxqP42IfxldewDqMsj67MclHS+uf2j7oKTlo24MQL0+03t225OSvibp98WmB22/bnu77UU99tlku2u7OzMzU6lZAMMbOOy2F0raJen7EXFG0s8kfVXSSs0e+X88334RsTUiOhHRmZiYqKFlAMMYKOy2v6DZoP8yIn4tSRFxIiLORsSfJf1c0qrRtQmgqkE+jbekbZIORsRP5mxfNudu35Z0oP72ANRlkE/jb5a0XtJ+2/uKbT+UtM72SkkhaVrS90bSIYBaDPJp/O8kzfc71M/X3w6AUeEbdEAShB1IgrADSRB2IAnCDiRB2IEkCDuQBGEHkiDsQBKEHUiCsANJEHYgCcIOJEHYgSQcEeMbzJ6R9L9zNi2WdGpsDXw2be2trX1J9DasOnu7JiLm/f23sYb9U4Pb3YjoNNZAibb21ta+JHob1rh642U8kARhB5JoOuxbGx6/TFt7a2tfEr0Nayy9NfqeHcD4NH1kBzAmhB1IopGw215j+03bh2w/3EQPvdietr2/WIa623Av222ftH1gzrYrbL9o++3ict419hrqrRXLeJcsM97oc9f08udjf89ue4GktyT9g6Sjkl6RtC4i/nusjfRge1pSJyIa/wKG7dWS/ihpZ0T8TbHtnyWdjogtxX+UiyLioZb09qikPza9jHexWtGyucuMS7pb0j+qweeupK+1GsPz1sSRfZWkQxHxTkT8SdKvJN3VQB+tFxEvSzp9wea7JO0oru/Q7D+WsevRWytExPGIeK24/qGkc8uMN/rclfQ1Fk2EfbmkI3NuH1W71nsPSb+x/artTU03M4+lEXFcmv3HI2lJw/1cqO8y3uN0wTLjrXnuhln+vKomwj7fUlJtmv+7OSK+Luk2SQ8UL1cxmIGW8R6XeZYZb4Vhlz+vqomwH5V09ZzbX5Z0rIE+5hURx4rLk5KeUfuWoj5xbgXd4vJkw/38vzYt4z3fMuNqwXPX5PLnTYT9FUnX2v6K7S9K+q6k5xro41NsX1Z8cCLbl0n6ltq3FPVzkjYW1zdKerbBXs7TlmW8ey0zroafu8aXP4+Isf9Jul2zn8j/j6R/aqKHHn39laT/Kv7eaLo3SU9r9mXdx5p9RXSfpCsl7ZH0dnF5RYt6+w9J+yW9rtlgLWuot7/T7FvD1yXtK/5ub/q5K+lrLM8bX5cFkuAbdEAShB1IgrADSRB2IAnCDiRB2IEkCDuQxP8Bn1MyZC6pCMUAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "# 图片展示\n",
    "%matplotlib inline\n",
    "some_digit=X_train[40000]\n",
    "some_digit_img=some_digit.reshape(28,28)\n",
    "plt.imshow(some_digit_img,cmap=matplotlib.cm.binary)\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [],
   "source": [
    "def image_offset(data_s,dir='u',offset=1):\n",
    "    data_r=data_s.reshape((len(data_s),int(np.sqrt(data_s.shape[1])),-1))\n",
    "    data_new=[]\n",
    "    if dir=='u':\n",
    "        for img_data in data_r:\n",
    "            img_data_new=np.concatenate((img_data[offset:,:],img_data[:offset,:]),axis=0)\n",
    "            data_new.append(img_data_new)\n",
    "        \n",
    "    elif dir=='d':\n",
    "        for img_data in data_r:\n",
    "            img_data_new=np.concatenate((img_data[(len(img_data)-offset):,:],img_data[:(len(img_data)-offset),:]),axis=0)\n",
    "            data_new.append(img_data_new)\n",
    "            \n",
    "    elif dir=='l':\n",
    "        for img_data in data_r:\n",
    "            img_data_new=np.concatenate((img_data[:,offset:],img_data[:,:offset]),axis=1)\n",
    "            data_new.append(img_data_new)\n",
    "            \n",
    "    elif dir=='r':\n",
    "        for img_data in data_r:\n",
    "            img_data_new=np.concatenate((img_data[:,(len(img_data)-offset):],img_data[:,:(len(img_data)-offset)]),axis=1)\n",
    "            data_new.append(img_data_new)\n",
    "    data_new=np.array(data_new)        \n",
    "    X_train=data_new.reshape(len(data_s),-1)\n",
    "    return X_train"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [],
   "source": [
    "#设置X_train移位参数\n",
    "X_train_u=image_offset(X_train,dir='u')\n",
    "X_train_d=image_offset(X_train,dir='d')\n",
    "X_train_l=image_offset(X_train,dir='l')\n",
    "X_train_r=image_offset(X_train,dir='r')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [],
   "source": [
    "X_train_new=np.concatenate((X_train,X_train_u,X_train_d,X_train_l,X_train_r),axis=0)\n",
    "y_train_new=np.concatenate((y_train,y_train,y_train,y_train,y_train),axis=0)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [],
   "source": [
    "# 实例化 \n",
    "from sklearn.neighbors import KNeighborsClassifier\n",
    "kn_clf=KNeighborsClassifier()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "KNeighborsClassifier(algorithm='auto', leaf_size=30, metric='minkowski',\n",
       "           metric_params=None, n_jobs=1, n_neighbors=5, p=2,\n",
       "           weights='uniform')"
      ]
     },
     "execution_count": 16,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# 填充数据\n",
    "kn_clf.fit(X_train,y_train)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([5.])"
      ]
     },
     "execution_count": 15,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# 验证\n",
    "kn_clf.predict([some_digit])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [],
   "source": [
    "from sklearn.metrics import precision_score,recall_score"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [],
   "source": [
    "from sklearn.model_selection import cross_val_score"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [],
   "source": [
    "from sklearn.model_selection import cross_val_predict"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([False, False, False, ..., False, False,  True])"
      ]
     },
     "execution_count": 20,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "y_train_5=(y_train.astype(int)==5)\n",
    "y_train_5"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "KNeighborsClassifier(algorithm='auto', leaf_size=30, metric='minkowski',\n",
       "           metric_params=None, n_jobs=1, n_neighbors=5, p=2,\n",
       "           weights='uniform')"
      ]
     },
     "execution_count": 21,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "kn_clf.fit(X_train,y_train_5)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "y_train_pred=cross_val_predict(kn_clf,X_train,y_train_5,cv=3)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# 精度，召回\n",
    "precision=precision_score(y_train_5,y_train_pred)\n",
    "recall=recall_score(y_train_5,y_train_pred)\n",
    "print('precision:{:.1f}%'.format(precision*100))\n",
    "print('recall:{:.1f}%'.format(recall*100))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "kn_clf_new=KNeighborsClassifier()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "y_train_new_5=(y_train_new.astype(int)==5)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "KNeighborsClassifier(algorithm='auto', leaf_size=30, metric='minkowski',\n",
       "           metric_params=None, n_jobs=1, n_neighbors=5, p=2,\n",
       "           weights='uniform')"
      ]
     },
     "execution_count": 27,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "kn_clf_new.fit(X_train_new,y_train_new_5)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "kn_clf_new.predict([some_digit])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "y_train_new_pred=cross_val_predict(kn_clf_new,X_train_new,y_train_new_5,cv=3)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "precision=precision_score(y_train_new_5,y_train_new_pred)\n",
    "recall=recall_score(y_train_new_5,y_train_new_pred)\n",
    "print('precision:{:.1f}%'.format(precision*100))\n",
    "print('recall:{:.1f}%'.format(recall*100))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "y_test_5=(y_test.astype(int)==5)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "y_test_pred=cross_val_predict(kn_clf_new,X_test,y_test_5,cv=3)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "precision=precision_score(y_test_5,y_test_pred)\n",
    "recall=recall_score(y_test_5,y_test_pred)\n",
    "print('precision:{:.1f}%'.format(precision*100))\n",
    "print('recall:{:.1f}%'.format(recall*100))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### 结论\n",
    "- 没有进行数据增广的表现：\n",
    "    - precision:97.0%\n",
    "    - recall:95.5%\n",
    "- 像素位移后，模型表现\n",
    "    - precision:98.0%\n",
    "    - recall:97.2%\n",
    "- 测试集表现\n",
    "    - precision:95.3%\n",
    "    - recall:91.1%\n"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.5"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}
